NSF PAR Search | NSF Public Access Repository

Abstract This paper presents a conformal inference method for out-of-distribution testing that leverages side information from labelled outliers, which are commonly underutilized or even discarded by conventional conformal p-values. This solution is practical and blends inductive and transductive inference strategies to adaptively weight conformal p-values, while also automatically leveraging the most powerful model from a collection of one-class and binary classifiers. Further, this approach leads to rigorous false discovery rate control in multiple testing when combined with a conditional calibration strategy. Extensive numerical simulations show that the proposed method outperforms existing approaches.

Conformal Inference is (almost) Free for Neural Networks Trained with Early Stopping

Liang, Ziyi; Zhou, Yanfei; Sesia, Matteo (December 2023, Proceedings of the 40 th International Conference on Machine Learning)

Early stopping based on hold-out data is a popular regularization technique designed to mitigate overfitting and increase the predictive accuracy of neural networks. Models trained with early stopping often provide relatively accurate predictions,but they generally still lack precise statistical guarantees unless they are further calibrated using independent hold-out data. This paper addresses the above limitation with conformalized early stopping: a novel method that combines early stopping with conformal calibration while efficiently recycling the same hold-out data. This leads to models that are both accurate and able to provide exact predictive inferences without multiple data splits nor overly conservative adjustments. Practical implementations are developed for different learning tasks—outlier detection, multi-class classification, regression—and their competitive performance is demonstrated on real data.

Full Text Available

Search for: All records